Skip to main content
W&B Fully Connected
>
Articles

Machine Learning Hackathon: Detecting Filters in Images

Using 40,000 images to train a Convolutional Neural Network on detecting specific filters in new images
Lachie N
Created on February 11|Last edited on March 15
Note: This was a notable post during our recent hackathon with Replit. You can view the associated Repl here.
💡

The idea

Last week on Sunday at 3 AM (AEST), the first Replit x Weights and Biases ML began. That same morning, I began creating my project, on a shiny new 36x boosted Repl.
The idea of detecting filters didn't come out of nowhere though. The past few days before the kickoff, I was brainstorming ideas of what I wanted to do.
The first attribute I decided is that I wanted to do something with images. I had been watching a few CaryKH videos whilst brainstorming, and really liked what he did with models like HyperGAN and autoencoders.
After spending way too long having a think of what I could do, I settled on 2 ideas I could try:
  1. Autocomplete for handwriting using HyperGAN and an extension called Pix2Pix
  2. Detecting filters on images and using JIMP (JavaScript image manipulation tool) to reverse them
I couldn't decide, however, and frankly, I didn't know if my initial first idea of handwriting autocomplete was doable in the short time frame.

Consulting PhD. ChatGPT

To settle on an idea, I used OpenAI's ChatGPT, talking to it about my ideas and learning about Tensorflow.js in general.
The first question of many I handed to my AI overlord
Learning about convolutional neural networks
Learning that my first idea wouldn't be doable in the time frame, and would be too much effort.
Using ChatGPT, I narrowed down my ideas and settled filter removal. The first one would simply take too much time and computing power to complete, and would be extremely hard for a beginner in ML like me.

The idea–exactly–and the training data

What I wanted to accomplish was clear, and I set out on a roadmap to getting the project completed:
  1. Find a source where I can easily scrape thousands of images & apply filters to them to give to a Convolutional Neural Network
  2. Train the CNN on these thousands of images, and reward it depending on how close it got to predicting what filters went into the images
  3. Run tons of experiments with W&B to get just the right hyperparameters
  4. Create the final run & an interactive website to test it
  5. Profit

Thousands of images, where?

To train a CNN, I'd need training data. A batch of images that would be swapped every 60 epochs or so, with random filters applied.
To retrieve the images, I used the Pixabay.com API. It has an extremely generous license and hundreds of thousands of images, allowing me to efficiently get random copyright-free images from the internet. I could then apply filters to the images with JIMP, and use the number of filters I applied as labels for the model to train towards.
During most of my experiments, I logged sample training data:


The images were all 64x64, with some of the following filters applied:
  • Brightness
  • Contrast
  • Saturation
  • Tint
  • Shade
  • Extra red
  • Extra green
  • Extra blue

The first batch of experiments and the first batch of mistakes

After around 2 days of getting myself comfy with Tensorflow.js, I started running 13 experiments overnight, logging them on wandb for me to check in the morning.
I had the experiments in JSON format:
{"experiment":true,"currentExperiment":17,"experiments":[{"name":"normalSettings","batchSize":55,"trainingDataChanges":1,"layerUnits":128},{"name":"heavierTraining","batchSize":55,"trainingDataChanges":1.25,"layerUnits":128},{"name":"heaviestTraining","batchSize":55,"trainingDataChanges":1.5,"layerUnits":128},{"name":"lighterTraining","batchSize":55,"trainingDataChanges":0.85,"layerUnits":128},{"name":"lightestTraining","batchSize":55,"trainingDataChanges":0.5,"layerUnits":128},{"name":"batchSize80","batchSize":80,"trainingDataChanges":1,"layerUnits":128},{"name":"batchSize30","batchSize":30,"trainingDataChanges":1,"layerUnits":128},{"name":"moreUnits","batchSize":55,"trainingDataChanges":1,"layerUnits":164},{"name":"tonsOfUnits","batchSize":55,"trainingDataChanges":1,"layerUnits":216},{"name":"lessUnits","batchSize":55,"trainingDataChanges":1,"layerUnits":86},{"name":"veryLowUnits","batchSize":55,"trainingDataChanges":1,"layerUnits":42},{"name":"heaviest","batchSize":80,"trainingDataChanges":1.5,"layerUnits":216},{"name":"lightest","batchSize":30,"trainingDataChanges":0.5,"layerUnits":42}]}
...and then I woke up to find out they had crashed. I was disappointed, but it wasn't the last time something like that would happen.
Around 4 days into the competition, I finally got it running. Here were the results:


Little did I know, the base model I was using was extremely flawed. While debugging a NaN loss issue on the 2nd day, ChatGPT mistakenly gave me a bad model structure, and I used it.
Nonetheless, it got me a bit of guidance on what I should and shouldn't be doing, such as the fact that lighter training data made the loss go lower, but the accuracy plummet too, and harder training data had the loss be very high, but the accuracy is the best.

It works! Just a few more mistakes and it'll be good...

Ecstatic that it had finally worked, I started the first long training experiment of many. This was meant to eventually become the final product, but as there's a longtrainingv5 experiment, you'd know it wasn't smooth.


This ran for 9 hours. You'd expect to have an improvement, but no, the accuracy and loss stayed the same.
After school on the very same day, I launched v2 of my long experiment batch.


Finally, it had changed! Unfortunately not for the better.
As you can see, not everything was smooth in v2. I tried tweaking a few small parameters here and there, but it just kept getting worse.
One good thing that did happen though was that I got a static valuation dataset, to track the experiment's performance better. This was suggested by Darek Kłeczek on Discord, along with him giving me great feedback too. Thanks, Darek!

The mistakes never end!

Sure, this was my first machine learning hackathon but I didn't expect this. I had tried 4 long experiments so far, and they didn't seem to be helping.


I can't tell you why it didn't work the first 4 times (or 17 times if you count the initial experiments), or why the valuation accuracy randomly spiked to the tens of thousands, but I can show you this:

Absolute confusion
ERR_NOT_SATISFIED

The last big mistake - an obscure website oversight

Around the time of training v4 (you know it's not the last), I spent an hour and a Digitech lesson on creating a website to test and review the model on different, never-before-seen images. It looks like this:

This looked great, and I was really excited to see it come together! However, when I added the filters detected section, I got a sinking feeling. For every image I gave it, the filters detected were the exact same.
Start half an hour of panic, fretting my model wasn't learning anything and just guessing. ChatGPT wasn't helping either.

Me and ChatGPT hadn't the slightest clue about what was happening, and I was worried that it was the model and I had everything wrong. Luckily though, I was wrong. the program just wasn't letting the image load in fully before evaluating it.
Fixing it made me realize how bad my current model was going, however, so I decided to start a v5, with an extremely beasty model to use as much power as my 16x Repl could provide.

Version 5 and the results



Now we're talking. I ran v5 to great results. It got to its best very quickly, and after training it for 2 days and 2 nights, I waited for the right moment to stop it for the final product. It had the highest accuracy and the lowest loss and was astonishing at removing filters.
Here are some examples I'd like to share:
  1. Random cat on Instagram

2. A lakeside landscape

3. Shane, a Replit staff member

4. My iconic muffin profile picture

The original muffin without the filter

Conclusion, thank yous, and how to try it yourself

This past week has been an absolute joy for me. I started off knowing literally nothing about AI, what a Convolutional Neural Network was or how to even begin. ChatGPT, a language model by Open AI, actually helped me the most in this, which honestly scares me for the future.
In the end, the model is able to accurately detect and remove filters to the point where the output is closer to the original than the filtered. All the source code is open source, so if you'd like to train it yourself you can 100% go for it.
If you'd like to test it yourself, the website is public & published on Replit. You can view it here.
Before I go, I'd like to say thanks to kind folks in the Replit Discord who helped me out:
  • Rayhanadev for moral support, feedback & wisdom
  • Icemaster Eric for feedback & goofiness
  • Ornella for making the event possible and making sure everyone's on the same page about the event details
  • Darek Kłeczek for technical feedback & vaulation dataset suggestion
  • Mario Vertinski & Haroon for moral support
  • Everyone from Replit and Weights & Biases for making this possible!
Kaz Brekker
Kaz Brekker •  
Nice job on the report. Honestly I would have never been able to do this as fast as you did.
Reply
Morgan McGuire
Morgan McGuire •  
That was a great read :)
Reply
Ayush Thakur
Ayush Thakur •  
Loved the report. Great job. :)
Reply
Tags: Articles, Community Posts, Beginner, Computer Vision, Experiment

Created with ❤️ on Weights & Biases.

https://wandb.ai/coding398/Image%20Classifier/reports/Machine-Learning-Hackathon-Detecting-Filters-in-Images--VmlldzozNTM1OTQ3
Made with Weights & Biases. Sign up or log in to create reports like this one.
Iterate on AI agents and models faster. Try Weights & Biases today.
Sign upTry W&B now